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ABSTRACT 



The invention relates to a method and device for detecting and 
analyzing the reception behavior of people in at last one reception 
room according to visual, audiovisual and/or auditory messages, 
actions and/or to the guiding of flows of customers. The method should 
be able to be realized with a lower degree of complexity than that of 
prior art methods. The invention is characterized in that, fix)m the 
moment each person enters the reception room until their departure, 
their location coordinates, body coordinates, turning movements and 
rotating movements of their body and/or of parts of their body, as well 
as the posture of their body and/or the position of their extremities are 
detected with a frequency that is greater than one. 
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METHOD AND ARRANGEMENT FOR DETECTING AND 
ANALYZING THE RECEPTION BEHAVIOR OF PEOPLE 

BACKGROUND OF THE INVENTION 

5 The invention relates to a method and arrangement for detecting and 
analyzing the reception behavior of people in at least one reception room 
in dependence on messages transmitted visually, audiovisiially and/or 
auditorily, according to the type of claims. In particular, the invention 
relates to the automatically and objectively verifiable detection of the 

10 advertising effectiveness of messages, activities and facilities for guidmg 
flows of customers, as well as for detecting the intentional customer 
behavior *m shops, shopping streets, shopping arcades, terminals, railway 
stations, petrol stations etc. 

15 For detecting the effectiveness of advertising measures for products and 
services, sensors such as infixed sensors, CCDs and the like are used in 
a suitable arrangement in order to count persons on entering an advertised 
range and on departing from the same, as well as to detect more or less 
statistically how long they stay in the advertised range. If necessary, also 

20 the biQing behavior of the persons in dependence on an advertising 
campaign, which is on or not, can be indirectly detected by detecting the 
actually bought goods via a registration by the electronic cash at the exit 
of an advertising range and a purchasing area, respectively. The results 
obtained can be evaluated in a computer in order to draw certain 

25 conclusions from the buying behavior of the recipients in dependence on 
advertising. This method, however, is not veiy precise in its presently 
practiced form since, for example, the mere presence in an advertised 
range does not tell anything on the attention paid to the advertising. 
A precise method for detecting the perception of the visual or audiovisual 

30 messages is the so-called "eye-tracking', which detects and registers the 
eye movements of one or a few person/s by means of suitable sensors 
provided at suitable locations. The position of the eye and of the pupil 
signalizes the perception of and the attention to the advertising 
instruments by the respective recipient. The relation between the 

35 perception of and the attention to, respectively, the advertising and the 
buying behavior of the recipient can be foimd out by a long analyzing 
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chain in combination with interviews. This method is technologically very 
expensive when one considers that, inter alia, a plurality of recipients is 
staying in an advertising and shopping range, whereby the movements of 
the eyes of each recipient have to be tracked. 

5 Furthermore, there are various methods known for selecting persons in 
and from video images, as well as for detecting head and shoulder 
formations and for detecting and storing feces. They are used for counting 
and identifying as well as for verifying the presence of persons. Thereby 
head or shoulders, among others, are generally identified and body 

10 models are associated thereto. These modeled bodies and the body 
coordinates resulting therefrom are, to the largest extent, virtual ones and 
they are not suitable for a real attention analysis and communication 
analysis. 

15 SUMMARY OF THE INVENTION 

It is an object of the present invention to provide a precise method, which 
is of low e^qpenditures, for detecting, analyzing, and evaluating the 
movements of persons to find out the advertising efficiency of messages, 
actions and customer flow guiding, and an arrangement for carrying out 

20 the same. 

According to the invention, the object is realized by the features of the 
first and the fourth generic claim, which are advantageously embodied by 
the features of the dependent claims. 

25 The invention is characterized in that a definite room is repeatedly 
detected in its entirety and its details by aid of at least one passive or 
active sensor at a conq)aratively high frequency. The detection can be 
combined with a counting of the persons and/or with an identification of 
the same. The detection firequency shall be greater unity, it will 

30 advantageously be 7 through 30 Hz. The sensor/s can be a CCD-camera 
(passive) or a laser or other sensors emitting and detecting, respectively, 
electromagnetic or acoustic waves and thermal radiation, respectively, 
(active and passive). It/they can be located at the ceiling or at the 
sidewalk in the upper range of the defined room. When an 

35 electromagnetic or acoustic or thermal transmitter is used, then the 
contents of the room is scaimed in a rapid sequence and in this way the 
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profile across the ground plan of the room is actively generated including 
the objects and subjects, respectively, which are present in the room 
whereas, when using a CCD-camera the contents of the room is imaged 
on a CCD-matrix which itself is passive as concerns the imaging process. 

5 Anyhow, not only all changes in the room can be re^stered and the tracks 
of the persons and the objects moving in the room finom their entry into 
the room up to their exit therefiom, but also the turns of the body and of 
the head of a respective person and ttie intentional turns, the changes in 
their posture and the posture of their extreinities, respectively, can be time 

10 dependently detected, so can the speed and the staying times of the 
individual objects and subjects. Non-moving objects will also be 
registered, but they will be eliminated fit)m the further data processing or 
rather will not be considered thereby. The contents of the room can be 
detected by a single-image method or by a two-image method 

15 (stereoscopically). Furthermore, it will be possible to evaluate not only 
black-and-white images but also colored images. Furthermore it is 
possible to, for example, spectroscopically evaluate not only optically 
produced images and mirrored images, respectively, but also such having 
been generated by radio or thermal radiation. In this way it is abo 

20 possible to detect objects and subjects by their size, gender and age 
provided that they are in the defined room. 

According to the invention preferably means and methods (algorithm) of 
the image processing are used for detecting and identifying objects and 

25 subjects being present, respectively, moving in the defined room. 
Concentration points are deducted from gray values and chromatical 
values, respectively, or firom characterizing, preferably circumscribing, 
polygons, including triangles. The moving behavior of persons and 
objects will be deducted from &e time^lependent position variations of 

30 said concentration points. Also differential concentration points can be 
formed for detecting the number, position, direction of movement, 
patterns of movement, speeds, staying times, body postures and body 
turns of the persons, as well as their lines of sight Such differential 
concentration points can be captured in a Cartesian spatial coordinate 

35 system or by vectors and wUl be processed under inclusion of prominent 
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points or lines of the body, of the extremities of the same, of shoulders 
and/or the head of persons. 

In contrast to the known methods, the "mventional method permits a higher 
counting precision since the counting is carried out in the room and not 
5 along a barrier (line, plane). A further differentiation of the persons 
captured will be possible by the spectroscopic and/or thermal analysis of 
the radiation and waves, respectively, reflected at said persons. 

An arrai^ement for detecting number and movements, features and 

10 coordinates of objects and subjects in dependence on the oflfered 
messages comprises in principle one or a plurality of sensor/s (sensor 
modules) or camera/s (camera modules) that is/are arranged at the ceiling 
and/or at the sidewalls, in particular in the upper range of a defined room 
near the ceiling. It further comprises image processing units downstream 

15 of said sensors/cameras, whereby said image processing units comprise 
fiame grabbers, socket mterfeces, Ethernet interfeces and other remote 
data transmission interfeces, respectively, and a database module (with 
socket interface and Ethernet interfece, respectively). The individual 
database modules can be connected to a central evaluation and/or 

20 instruction module via remote data transmission, whereby said central 
evaluation and/or instmction module forms the results from the signals 
provided by the sensors and is capable of influencing and controlling the 
messages, actions, and customer guiding systems offered in the defined 
rooms. Each sensor module and camera module, respectively, is followed 

25 by a sensor signal module and an hnage processing module, respectively, 
which captures each object or subject entering the defined room and 
identifies the same as a such one and differentiates the same, where 
appropriate. Due to the repeated detection or e}q>osure of the room and its 
contents, they detect in the manner as desnibed hereinbefore the track of 

30 the object/subject, its movements in and through the room to exiting 
the roorn, as well as its specific features obtained in this way, and they 
feed the results achieved in this way to the evaluating database module. 
Thereby one database module can belong to a plurality of sensor modules 
and image processing modules. 

35 In interaction between the sensorinfonnation processing and the image 
processing, respectively, and the statistically evaluating database, the 
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sensor module and the image processing module, respectively, is capable 
of constantly generating a plurality of data. The analysis of said data in 
the database module delivers, optionally and simultaneously, respectively, 
information on the customer flow and the recipient behavior up to the 

5 individual customer, whereby these data can be quantified and qualified. 
If appropriate and on request of the image projection for the advertising 
display, the sensor module and the im^e processing module, 
respectively, can be re-started, interrogated and/or the information of the 
same detected with each new advertising spot In the first and the second 

10 case described hereinbefore, the time-related and spot-related respectively 
contents-related analysis of the recipient behavior is possible. In the 
second case mentioned hereinbefore, only a spot-related analysis of the 
recipient behavior can be carried out as a whole. Provided that the spot 
can be subdivided into intervals, again a time-related and sequence- 

15 related analysis is possible, which permits conclusions as concerns the 
contents of the spots and their structure. When, however, the spot analysis 
is subdivided into intervals, then it is possible to optimize the contents of 
the spots and the structure of the spots because of the effectiveness and 
attention, respectively, which the individual sequences of the advertising 

20 messE^es have achieved. Due to the high image analyzing fi-equency, an 
interior decomposition of the spots is possible. Since, for example, with a 
spot lasting for 15 second at a scanning fi^quency of 7.5 Hz, more than 
100 images are available for the analysis, whereby said images can be 
decomposed in a plurality of spot intervals, for example, into five 

25 reasonable spot intervals, the advertising effectiveness of the individual 
scenes can be very well and exactly detected therefirom. 

DETAILED DESCRIPTION OF THE INVENTION 

In the following, the invention will be e?q>lained in more detail by virtue 
30 of the schematical drawings. There is shown in: 
Fig. 1 a passage with three defined rooms. 
Fig. 2 the effect of the central perspective of the camera on the 

detected room. 
Fig. 3 the image of a reception room on a CCD-matrix of a 
35 camera. 
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Fig. 4 the principle layout of the arrangement according to the 
invention, and 

Fig. 5 a combination of a plurality of inventional arrangements 
with a central unit for evaluation and providing commands. 

5 

In Fig. 1 different objects and subjects (recipients) r are represented 
executing diflferent movements and directions of movements in defined 
reception rooms Wi through W3 of a passage P. The differences in the 
objects and subjects are exemplified by their differentiated reflection 

10 behavior which, in tum, is due to size, clothing, body form, age, gender, 
persons and matter taken along by them, etc. which are here summarized 
imder the term of characteristic or specific features. A display d is 
arranged at a sidewall s of the reception room W2- A camera module Cj 
respectively C2 respectively C3 is associated to each reception room Wi 

15 through W3. In accordance with a preselected time schedule, said camera 
module detects the contents of its reception n>om and feeds it into a not 
shovm evaluation unit In this way it is possible to detect the persons as 
recipients r by number and differentiation, their movements, turns of the 
body, movements of head and arms, their stops, and to conclude 

20 therefi-om the attention they pay to the advertising messages on the 
display. By the overlapping ranges iii, the passing over of tiie persons 
identified by coordinates and characteristic features to a following 
evaluation unit (not shown in Fig. 1), which is succeeding the camera 
module Cj, will be possible (range overlapping tracking), said evaluation 

25 unit will be described in the following. 

Fig. 2 again shows a reception room w covered by a sensor module c, and 
a display d arranged at a sidewall s of said reception room w. The sensor 
c is coupled to a bio-sensor c^ which also can be an integral component of 

30 the sensor c and which evaluates spectroscopically or radiologically the 
radiation features differentially reflected at the recipient r. In said 
reception room w there are several recipients r that move into diflferent 
directions respectively, stop and stay at definite places. A space 
circumscribed by the dotted line 1 makes clear that only the recipients 

35 (objects and subjects, respectively) r in this space are completely 
captured. Hence, the groimd plans of the mutually adjacent reception 
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rooms Wi have to overlap due to the central perspective of the optical 
imaging and to the end that the identified persons have to be taken over, 
so that all the recipients, with respect to their movements and specific 
features, can be completely detected up to a height of about 2 m. 

5 

In Fig, 3 there is shown a CCD-matrix m of a sensor module c with a 
sufQcient resolution. On said sensor module c both, a non-displaceable 
sidewall s, provided with a display d, is represented and recipients r as 
pixel blots aj. The pbcel blots are provided as gray values conditioned by 

10 the respective persons, environment, and movement, whereby the 
recipients r move on track bi. Thereby a plurality of exposures are 
superimposed, which are taken at a fi^uency of, for example, 25 Hz and 
firom which, for example, every fourth and second, respectively, is to be 
used for evaluation. It can clearly be seen that the recipient r 

15 corresponding to the pbcel blot ai has moved with a greater speed than the 
recipients Vi represented by the pixel blots a2, aa, a4, fi-om these only the 
recipient represented by the pbcel blot as has turned to and feces the 
display. Furthermore, it can be seen that the recipients represented by the 
pbcel blots a2, as, a4 have moved with very differing speeds at different 

20 times, whereby the recipient represented by the pbcel blot as indicates 
his/her being interested in tfie displayed message in that he/she 
remaricably reduces the speed down to a stop. The recipient represented 
by the pbcel blot aj and moving on the track bs, only tangiently passes the 
defined and imaged room. 

25 

In Fig. 4 similar to Fig. 1 three sensor modules Ci, C2, Cj are represented 
followed by three image processing modtiles pi and a common database 
module tl The image processing modules pi constantly generate a 
, plurality of data, by the time dependent analyses of which in the database 
30 module n the number and behavior of the customers is achieved time- 
dependently. Instead of being arranged in spaced apart relation the sensor 
modules Ci and the image processing modules p; can be combined into a 
unit. 

Thus, the time-dependent analysis of the movements of persons and the 
35 specific features of persons will be combined with the messages or 
actions offered time-dependently visually, audiovisually and auditorily. 
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respectively. In this way the spot-related behavior and the action related 
behavior, respectively, of individual recipient can be analyzed as well as 
the flow of customers. The images generated in the sensor modules Cj will 
be processed in the image processing modules pi. Thereby the specific 

5 optical features (gray values or color values etc.) are detected for each 
person just as the concentration point coordinates of the body or of parts 
of the body (extremities, head, nose) and the coordinates of 
circumscribing polygons. An identification number is associated to each 
person detected in such a way. Furthermore, the speeds and the vectors of 

10 direction of the movements of persons will be detected in the image 
processing modules pi just as their mutual distances and their distance to 
the site of presentation. 

The data detected in this manner are fed into the database module n which 
therefi-om computes the movement behavior of the individual persons up 

15 to the stay time, it also deteraiines a mean velocity in defined action 
rooms or reception rooms, and the speed of the individual persons is 
related to the mean velocity. Thus, the database module n can detect 
direct and indirect criteria of attention, it fiirther can evaluate them as to 
their importance and can forai mean values. Direct criteria of attention are 

20 ori^nal ones, for example, coordinates, indirect criteria are deducted ones 
such as speed. 

Fig. 5 shows four module units gi, g2, g3, g4> which are differently 
equipped with camera processing modules and image processing modules. 

25 A respective database module nj, n2, ns, is associated to each module 
unit gi, g2> g3> g4, and all the database modules are connected to a central 
evaluation module and, if necessary, control module z, into which they 
feed their data and by which they are controlled, if required, spot- 
dependently. By ^nrtue of the analyzing data, the control module z is 

30 capable of optimizing the projection mode of the spots as concerns their 
sequence, position and fiequency. 

All features disclosed in the specification, in the subsequent claims, and in 
the drawings are substantial for the invention both, individually and in any 
35 combination with one another. 
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LIST OF REFERENCE NUMERALS 



ai pixel blot/s 

bi tracks 

5 c, Ci camera and sensor modules, respectively, 

Cb bio-sensor 

d display 

gi module units 

1 dotted line/line draw 

10 m CCD matrix 

n, ni database module/s 

ps image processing modules 

r recipient/s 

s sidewall/s 

15 ij/fli overlapping ranges 

w, Wi reception rooms 

z central evaluation module and control module 

P passage 
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CLAIMS 

1 . A method for detecting and analyzing the reception behavior of persons 
in at least one reception room in dependence on visual, audiovisual 
5 and/or auditory messages, actions and/or guiding of flows of 
customers, whereby the position coordinates of each person are 
detected time-dependently by a selectable frequency, which preferably 
is greater unity, characterized in that, in addition to the position 
coordinates of each person, the body coordinates of each person, 
10 her/his turns and rotations of her/his body or/and of parts of the body 
as well as her/his posture or/and the posture of her/his extremities are 
detected from her/his entry into the reception room \xp to her/his exit 
from the reception room. 

15 2. Method as claimed in claim 1, characterized in that the body 
coordinates comprise both, the concentration point coordinates of a 
projection of a person and of single parts of the body of a person as 
well as the outline coordinates of the person. 

20 3. Method as claimed in claim 1, characterized in that at least one 
identification value and respective coordinates are associated to a 
person, which are used for taking over said person when moving fix)m 
one reception room to a neighboring one. 

25 4. Arrangement for carrying out said method as claimed in at least one of 
the claims 1 through 3, characterized in that at least one sensor module 
is provided in tiie upper range of each reception room, said sensor 
module being adapted for detecting the electromagnetic radiation from 
the person present in said reception room, and in that the sensor 

30 module is followed by an image processing module and a database 
module. 

5. An arrangement as claimed in claim 4, characterized in that a plurality 
of sensor modules and image processing modules are pre-connected to 
. 35 said database module. 
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6. An arrangement as claimed in claim 5, characterized in that at least two 
database modules are connected to a central evaluation and control 
module, 

5 7- An arrangement as claimed in claim 4, characterized in that a further 
sensor module is associated to said one sensor module, said fiirther 
sensor module is adapted for feeding the electromagnetic or thermal 
radiation to a spectroscopic or thermal evaluation. 

10 8. An arrangement as claimed in claim 7, characterized in that said one 
sensor module is a CCD-camera. 

9. An arrangement as claimed in claim 4, characterized in that the sensor 
module is centrally arranged at the ceiling of the reception room. 

15 

10. An arrangement as claimed in claim 4, characterized in that adjacent 
reception rooms are overlapping each other. 
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NEW CLAIMS 

1. A method for detecting and analysing the reception behaviour of persons in at least one reception 
room in dependence of visual, audiovisual and/or auditory messages, actions or customer flow 
guides, whereby the position coordinates of each person are detected time dependently by a 
selectable frequency, which preferably is greater than 1. in the form of surface concentration 
point coordinates before their entry into the reception room and up to their exit from the 
reception room, characterized in that in addition to the surface concentration point coordinates 
further selected body coordinates of each person are detected, which include grey value 
concentration point coorduiates of the persons as well as differential concentration points of 
individual body parts of the persons, whereby the turns and rotations of the body of the person 
and/or of parts of the body as well as the posture of the body and/or of parts of the body are 
determined. 

2. Method according to claim 1, characterized in that the body coordinates include also the outline 
coordinates of the persons or parts of the bodies of the persons. 
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